Why do data platforms fail?

This is the first article in the three-part series. Read Part Two here and Part Three here.

Companies have spent untold millions on data projects over the years — all too often with unsatisfactory results. But why is it that even with the best data platform technology, the smartest data engineers and a leadership team committed to the cause, so many data projects fall flat? Because getting it right isn’t just about technology, it’s about people.

Data products, AI/ML models, and analytics are meaningless unless humans engage with them. Making data available alone doesn’t create impact and business value.

We continue to see expensive data projects with excellent technology teams that don’t deliver the intended value. Too often technology teams with little knowledge of the business create data products and then complain about adoption. Too often the business stakeholders don’t engage with data teams enough to share context and domain knowledge and then complain that the outputs are useful. 

A human-centered approach is critical to create data products that humans can and want to use. When data products are built on what customers want and need to get their job done better and to achieve business outcomes, these products not only get adopted but become essential to run the business. Data products that not only get adopted but become indispensable in people’s workflow and produce business value. 

In this series of blogs we’ll explore ideas around applying design thinking to data products and platforms. The data awesome framework, which we’ve developed enables enterprises to bring customer obsession in their journey to become data-led by ensuring that business stakeholders and technologists collaborate successfully to generate impact and business value with data. 

When data is captured or transformed inside tech black boxes, valuable information is lost during the process. Decisions, choices, reasons, outcomes of why has been done in a certain way, what was supposed to be inside data, and expectations about outcomes are few examples of all the valuable information (aka metadata) that was generated in post-it exercises, decision making meetings, analysis documents and conversations, that goes simply lost after just few months and easily leads to fear of change (aka legacy data). The data awesome framework gives you a way to provide context and narrative for the data. By opening the black box we ensure the purpose is not forgotten and stakeholders have the ability to monitor the decisions made so far and change their mind.

Demonstrating value to users

Thoughtworks’ first attempt to bring data thinking into our work started many years ago as an in-house built application named GoFigure. The application ended up being very popular and successful. It was used by every country for their business decisions. However, as the business became bigger and more complex, we had to reevaluate our approach to data.

A few years ago, we decided to strategically invest in setting up a data program to drive our internal operations to become data led. A team of experienced engineers, analysts and product owners was set up to build a strategy and a roadmap for the data-enabled Thoughtworks program. The team put a lot of effort into building a world-class engineering platform. However, once the early adopters were onboarded and the initial euphoria was over, we started to find it very hard to onboard more business leaders.

New users were reluctant to use our BI tool for their jobs. Even the early adopters started going back to the previous tool (GoFigure) or started switching back to the spreadsheets. After much discussion, debate, and deliberation we realized that we missed thinking about the most critical component during our program. The customer (i.e people that were actually in need of that data) and their needs.

To continue our data journey, we applied the data awesome framework to our own data platform.The new approach involved redesigning the data systems with a customer-first mindset. This led the team to build a new design thinking approach to our data initiative — a Data Awesome approach! 

Data Awesome Mindset and framework helped us build better empathy with our customers, a better understanding of the business context and a better dashboard for our users.The new dashboards were intuitive to use and provided key insights based on what users wanted to know and recommended actions they should take based on the information.

The first dashboard delivered with this approach went viral, usage went up to four times, feedback went from “I can’t make head or tail of it” to “I can’t work without it”, ‘I am ready to drop my spreadsheet’. A cross-functional team paired with our global head of supply to improve the existing dashboard on the demand and supply gap. The new dashboard was tailored to the needs of the staffing leads and was designed to fit into their workflow: prepare for the weekly supply planning review. It empowered them to manage supply with an ability to review internal assignments and beach more effectively. Particularly in a supply constrained situation. This translates into increasing utilization and revenue for the business and giving employees more opportunities.

Building a repeatable process

In the next part of this three-part series, we’ll be exploring the data awesome framework in detail. The framework empowers everyone working with data from business stakeholders to technologists (data analyst, business analysts, data engineers, experience designer, data strategists) to collaborate effectively to the creation of data products that deliver more value than just making data accessible, they empower people and become essential to run the business. It is a step-by-step repeatable process to drive the creation of enterprise data products that are usable, useful and valuable.